Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ED-2793] Fix installation issues #3903

Closed
wants to merge 75 commits into from

Conversation

santhosh-tg
Copy link
Collaborator

@santhosh-tg santhosh-tg commented Sep 29, 2023

Minor-EnhancementED-2793 Provision OCI infra to setup Sunbird Ed

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Type of change

Please choose appropriate options.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Enhancement (additive changes to improve performance)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes in the below checkboxes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Ran Test A
  • Ran Test B

Test Configuration:

  • Software versions:
  • Hardware versions:

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

VISHNUDAS-tunerlabs and others added 30 commits April 27, 2023 10:00
…rs (ED-94) (project-sunbird#3737)

* Devops changes for PII data stories

* env variable change

* env variable and kong api name changed: Review fix

* ED-94 Devops changes
Fix syntax for API getSolutionReportInformation
…syntax_fix_ml_reports

ml-reports env syntax error fix added
…Kafka-topic-Script-Change

programuser.info Kafka topic added to script
Updated the value and added new keys and Values
…6.0.0

#SB-0000 fix: updating the pdataid and version
LR-491 : Added ML PII New Data Product Job Id to the SubmitDataExhaust API
…fsetEnvKey

[ED-523] Added env key for timezone Difference
Issue #ED-0000 fix:added env variable to store default board value
santhosh-tg and others added 23 commits July 3, 2023 18:45
ED-2091 Updated the Loggers path in config.j2
…6.0.0

Issue #ED-2580 fix: Update default tenant
Issue fix ED-2586 'Diksha' text is appearing in help page and in SSO …
* Add missing var

* ED-2164: Add new var assessment service - #IQ-329

* Update dial service search var

* Fix sytax issue

* Onboard new apis lern-release-5.3.0
…nt variable (project-sunbird#3876)

* Issue #ED-2632 fix: removed unused env variables and added OCI endpoint variable

* Issue #ED-2632 cleanup: Removed Azure related variables

* Issue #ED-2632 cleanup: changed added storage account url
* envChange CSP(Azure)

* CSP env changes

* removed csp references

* env changes for core service

* default added

* added all variables

* env variable name change

* ENV changes

* Changes to env variables
* Update config.j2

Adding new variables for Survey data sources

* Update main.yml

Adding Variables

* Druid injestion spec update
…Public-Bucket

changed to public variables for cloud config
…_analytics_changes

CSP ML Analytics Changes
project-sunbird#3881)

* ED-1251 - Add OCI specific devops changes - superseeds PR 3817 , 3735 (project-sunbird#3858)

* added task and role for artifacts upload - oci oss

Signed-off-by: Deepak Devadathan <[email protected]>

* overwrite oss file if exists

Signed-off-by: Deepak Devadathan <[email protected]>

* added the switch for getting oci deployed image tag

Signed-off-by: Deepak Devadathan <[email protected]>

* added oci in the task name for getting image version

Signed-off-by: Deepak Devadathan <[email protected]>

* added helm values for oci for internal lb provisioning

Signed-off-by: Deepak Devadathan <[email protected]>

* getting image tag for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* added oci oss upload for certificate template

Signed-off-by: Deepak Devadathan <[email protected]>

* corrected typo in the role input variable

Signed-off-by: Deepak Devadathan <[email protected]>

* removed prefix flag from bulk upload

Signed-off-by: Deepak Devadathan <[email protected]>

* added oci oss task for chatbot upload

Signed-off-by: Deepak Devadathan <[email protected]>

* updated the prefix logic for folder upload to oss

Signed-off-by: Deepak Devadathan <[email protected]>

* uploadfaq to oci oss

Signed-off-by: Deepak Devadathan <[email protected]>

* added a temp debug

Signed-off-by: Deepak Devadathan <[email protected]>

* testing bulk upload logic

Signed-off-by: Deepak Devadathan <[email protected]>

* testing oss_path variable

Signed-off-by: Deepak Devadathan <[email protected]>

* testing with a defined prefix

Signed-off-by: Deepak Devadathan <[email protected]>

* removed debug line and oss_path value

Signed-off-by: Deepak Devadathan <[email protected]>

* added overwrite flag for bulk upload

Signed-off-by: Deepak Devadathan <[email protected]>

* oci oss changes for uploadschema

Signed-off-by: Deepak Devadathan <[email protected]>

* typo in the csp switch

Signed-off-by: Deepak Devadathan <[email protected]>

* oci oss changes for plugins upload

Signed-off-by: Deepak Devadathan <[email protected]>

* testing by adding privileged: true to init container

Signed-off-by: Deepak Devadathan <[email protected]>

* removed privileged=true..as it didnt help the deployment

Signed-off-by: Deepak Devadathan <[email protected]>

* deployed image check for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* added the switch for selecting api version

Signed-off-by: Deepak Devadathan <[email protected]>

* corrected the right task file for plugin del and upl for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* added a temporary debug

Signed-off-by: Deepak Devadathan <[email protected]>

* testing flow

Signed-off-by: Deepak Devadathan <[email protected]>

* debug line

Signed-off-by: Deepak Devadathan <[email protected]>

* testing

Signed-off-by: Deepak Devadathan <[email protected]>

* disable delete and repload plugin temp

Signed-off-by: Deepak Devadathan <[email protected]>

* testing

Signed-off-by: Deepak Devadathan <[email protected]>

* adding OCI values for player environments

* updated daemonset deployed image retrieval

Signed-off-by: Deepak Devadathan <[email protected]>

* added role for oci-cli

Signed-off-by: Deepak Devadathan <[email protected]>

* test home location

Signed-off-by: Deepak Devadathan <[email protected]>

* updated the oci cli install location

Signed-off-by: Deepak Devadathan <[email protected]>

* include option oci-cli in template

edit jenkins pipeline template to include oci_cli option

* updated oci related task for desktop faq upload

Signed-off-by: Deepak Devadathan <[email protected]>

* added the missing uploads for desktop faq upload

Signed-off-by: Deepak Devadathan <[email protected]>

* added CLOUD_STORAGE_ENDPOINT env variable

Signed-off-by: Deepak Devadathan <[email protected]>

* updated the ansible variable for cloud_storage_endpoint

Signed-off-by: Deepak Devadathan <[email protected]>

* updated env template for lms to add endpoint

Signed-off-by: Deepak Devadathan <[email protected]>

* corrected the typo in the template for lms

Signed-off-by: Deepak Devadathan <[email protected]>

* updated desktop-faq-upload for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* added uploadfaq for oci rc-schema

Signed-off-by: Deepak Devadathan <[email protected]>

* updated bucket name for oci upload for schema

Signed-off-by: Deepak Devadathan <[email protected]>

* updated bucket name for artifact upl and download for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* added oci oss block for asset upload

Signed-off-by: Deepak Devadathan <[email protected]>

* updated oci buket name for deploy plugins

Signed-off-by: Deepak Devadathan <[email protected]>

* added es snapshot for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* added es repository for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* added repository changes for log-es6 for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* removed access key and secret from es s3 repo settings

Signed-off-by: Deepak Devadathan <[email protected]>

* add s3 compatible repository setting for es backup

Signed-off-by: Deepak Devadathan <[email protected]>

* changed the repository type to s3
Signed-off-by: Deepak Devadathan <[email protected]>

* plugin repository-s3 for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* temporarily disabling no_log for trigage

Signed-off-by: Deepak Devadathan <[email protected]>

* used the logical or

Signed-off-by: Deepak Devadathan <[email protected]>

* reverted no_log

Signed-off-by: Deepak Devadathan <[email protected]>

* default client name based setting

Signed-off-by: Deepak Devadathan <[email protected]>

* added signer override

Signed-off-by: Deepak Devadathan <[email protected]>

* added s3 client configuration for oci

Signed-off-by: Deepak Devadathan <[email protected]>

* corrected elasticsearch.yml template

Signed-off-by: Deepak Devadathan <[email protected]>

* corrected the variable names in elasticsearch.yml

Signed-off-by: Deepak Devadathan <[email protected]>

* removed s3 compatiblity variables from elasticsearch.yaml

Signed-off-by: Deepak Devadathan <[email protected]>

* tsting snapshot backup

Signed-off-by: Deepak Devadathan <[email protected]>

* without path-style-acess

Signed-off-by: Deepak Devadathan <[email protected]>

* testing with home region

Signed-off-by: Deepak Devadathan <[email protected]>

* using ip of endpoint

Signed-off-by: Deepak Devadathan <[email protected]>

* install repository-s3 plugin for log-es

Signed-off-by: Deepak Devadathan <[email protected]>

* updated s3 settings values

Signed-off-by: Deepak Devadathan <[email protected]>

* re arranged the order

Signed-off-by: Deepak Devadathan <[email protected]>

* corrected the default values for sunbird_player.env

Signed-off-by: Deepak Devadathan <[email protected]>

* removed the redundant check to get deployed image

Signed-off-by: Deepak Devadathan <[email protected]>

* removed csp switch for deployed image retrival

Signed-off-by: Deepak Devadathan <[email protected]>

* common login for  getting deployed image

Signed-off-by: Deepak Devadathan <[email protected]>

* removed csp switch for getting deployed image

Signed-off-by: Deepak Devadathan <[email protected]>

* added the environment variable for instance principal for oci operation

Signed-off-by: Deepak Devadathan <[email protected]>

* Fixed indentation error

* Update oss-upload-batch-no-poll.yml

missing content type caused a LOT of problem

* Changes for postgres ssl configuration

* added the storage type for deployment

Signed-off-by: Deepak Devadathan <[email protected]>

* added cloud-store-proxy for cors reverse proxy

Signed-off-by: Deepak Devadathan <[email protected]>

* enabling relative base path in content API

* enabling relative path in content

* setting relative absolute path to true

* issue with npm version

* Fix for DU-417

* Moved the storage config to ansible variables

* typo fix

* updated oci oss config entries

* updated defaults

Signed-off-by: Deepak Devadathan <[email protected]>

* corrected values

Signed-off-by: Deepak Devadathan <[email protected]>

* updated kafka setup zookeeper

Signed-off-by: Deepak Devadathan <[email protected]>

* updated the generic defaults

Signed-off-by: Deepak Devadathan <[email protected]>

* added conditional check for pg ssl

Signed-off-by: Deepak Devadathan <[email protected]>

* corrected misplaced oci instead of aws

Signed-off-by: Deepak Devadathan <[email protected]>

* removed https

Signed-off-by: Deepak Devadathan <[email protected]>

* used variable for cloud_storage_url

Signed-off-by: Deepak Devadathan <[email protected]>

* removed duplicate entry

Signed-off-by: Deepak Devadathan <[email protected]>

* added default values for cloud_public_storage_proxy

Signed-off-by: Deepak Devadathan <[email protected]>

* added defaults to stack sunbird role

Signed-off-by: Deepak Devadathan <[email protected]>

* added genereic names

Signed-off-by: Deepak Devadathan <[email protected]>

* removed redundant file

Signed-off-by: Deepak Devadathan <[email protected]>

* removed redundant variable cloud_store_base_path_placeholder

Signed-off-by: Deepak Devadathan <[email protected]>

* removed redundant variable

Signed-off-by: Deepak Devadathan <[email protected]>

* added default value for cloud_public_storage_proxy

Signed-off-by: Deepak Devadathan <[email protected]>

* added sample reference value for cloud_public_storage_endpoint

Signed-off-by: Deepak Devadathan <[email protected]>

---------

Signed-off-by: Deepak Devadathan <[email protected]>
Co-authored-by: Kenneth Heung <[email protected]>
Co-authored-by: subhash_chandra_budde <[email protected]>
Co-authored-by: Ramya <[email protected]>
Co-authored-by: rjanart <[email protected]>

* Update sunbird_lms-service.env

* Resolving conflicts at sunbird_lms-service.env

---------

Signed-off-by: Deepak Devadathan <[email protected]>
Co-authored-by: Deepak Devadathan <[email protected]>
Co-authored-by: Kenneth Heung <[email protected]>
Co-authored-by: subhash_chandra_budde <[email protected]>
Co-authored-by: Ramya <[email protected]>
Co-authored-by: rjanart <[email protected]>
Co-authored-by: Santhosh Gandham <[email protected]>
* Issue #ED-2632 fix: removed unused env variables and added OCI endpoint variable

* Issue #ED-2632 cleanup: Removed Azure related variables

* Issue #ED-2632 cleanup: changed added storage account url

* Issue #ED-2632 fix: removed duplicate variables
* Updated vars for cron schedule

* Update main.yml
* Update Jenkins version

* Fix job parameter

* Fix error: You need to install jmespath prior to running json_query filter
* ED-2838 Release-6.0.0 new variables

* Added with comments

* ED-2838 jenkins josb update and new variables

* added storage endpoint detials
@santhosh-tg santhosh-tg force-pushed the ED-2793 branch 2 times, most recently from 0bbc22d to 9603c8e Compare September 29, 2023 05:16
@santhosh-tg santhosh-tg closed this Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.